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ABSTRACT 

In 1971-72, 334 students in 16 sections in beginning 
French, and in 1972-73, 331 students in 24 sections in the saoie 
course, rated their graduate student instructors on a 35-lteia scale. 
Student perfornance data on the first, Bidtern, and final 
departmental exaninatlons and on SAT-V (1972-73 only) were taken, and 
residual leirnlng gain (final exaa corrected for first exam) 
conputed. Cluster analysis yielded two intercorrelated clusters. Two 
scales with each item having unit weight were then developed as the 
student rating variables. A substantial negative correlation between 
the two rating scales and residual learning was replicated across the 
2 years. (MJM) 
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Both atudent ratings of Instructors and the amount students learn in 
a course, corrected for their initial learning (hence residual learning, or 
residual gain), are almost universally regarded as important criteria of 



^ Instructor success* Of these two criteria, student ratings are the more 
^ easily obtained and the more widely used. Studies using ei:udent residual 
achievement as the criterion variable require the use of multiple section 
• O coursea In which the same or very similar content ^s taught to students by 
/ ^ UJ different instructors. A uniform assessment of thie students ' knowledge or 
aptitude must be made prior to or early in the coutse and a uniform final 
assessment made at the end of the course, then the Initial assessment regressed 
on the flftal assessment to obtain residual achievement scores* This proc<idure 
is time consuming and tedious and is rarely carried out » Typically the assuiq>*- 
tlon is made that if students like or respond favorably to, an Instructor they 
learn well from him or her* It is this assumption which is tested in the 
studies reported herei 

Procedures 

Sample . In 1971-72, 334 students (99X of enrollees) in 16 sections of 
beginning, french fithd In 1972-73, 331 students (63X of enrolleea) in 24 sec-^ 
t ions in the same course, rated their graduate student instructors on a 35 
item scaU* Student performances on the first, mld«*term and final department 
examinations and on the SAT-V (1972-73 only) were taken and residual learning 
gain (final exam corrected for first exam) computed. 

Scale Development . Items representing each of the principal factors y 
from the student rating scales developed by Deshpande, Webb and Marks (1970)^ 
were discussed with the course director and subsequently taken to the French 
Department, Members of the department selected from each factor those items 
they believed most valid for ins true tors in thelt department > for a total of 
30 items, and added five additional Items specific to teaching beginning 
French. Following administration to the 1971-72 students in the final week 
of the course, the items were re-factored (principal components with varimax 
rotation) but an interpretable multi-factor solution could not be attained. 
A subsequent cluster analysis produced one large cluster and one small clus- 
. ter closely correlated to it. Two subscales with each item having unit 

\ weight were then developed as the student rating variables. These scales 
y were also used in the 1972-73 replication. 

i ^ *The author, wish to acknowledge the extended cooperation of tha Dapartmant 

of FrancH and Italian and tha support of tha Research Con&itt«a on teaching 
c-^ and U4f|irt|f Indiana Univeralky, - 

?Moat of items in theaa acalaa ara rpoaifeiva" in ^ording> To avoid position 
habltlt a^ai reapoi^^^^ nagatlvdly. Nagatlval^ ^plllad 

iu<ad' wafa fevaraed In ISoJa lof"^^^^^ analysis. 
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The median Iftter-observer reliability for random samples of six students 
from each section represented in the 1971-72 sample was, for the total scale, 
.93. The reliability (Cronbach'a Alpha) of the total scale was .90 for . 
subscale 1, .88 and for subscale 2, .64. 

Test Development . Three teams, drawn from the instructors in the 
course, including the course director, made up the first, mid-term and final 
examinations, representing in the items the content covered in the course ^ 
syllabus and in class. These examinations were administered on the same day by 
each Instructor during the 3rd, 7th and last (15th) week of the course respectlv 
The first test covered grammar, and the remaining two grammar, dictation, 
composition and reading comprehension. 

To develop the achievement criterion variable for the 1971-72 group, 
the first examination score was regressed on the final examination score 
<r»«.47) using each student as one degree of freedom and a residual gain score 
for each student obtained. Subsequently, section means for the atudent rating 
scales, the first, mid-term and final examinations, and for residual gain, 
were calculated. These variables were then correlated with each other using 
each section as one degree of freedom. Identical procedures were used with 
the 1972-73 replication. In this replication the SAT-V scores were also 
correlated to the final exam (r- . 23) , to test it as a possible covarlate. 
It was rejected as a useful covariate. 

Results 

The items appearing in each of the two subscales are shown in Tfbl6 1. 
The items in subscale 1 were originally associated with several first order 
factors extracted by Deshpande, Webb and Karks', Including structure, rapport, 
motivation overload and content mastery. In the present analysis, however, 
these Items seem to align with three x>t the four second order factors extracted 
by these authors, namely, cognitive merit, affective merit and stress. After 
Ipspectlng the items, this subscale was thus labeled "Instructor cognitive 
and affective merit versus student cognitive and affective stress. Unlike 
subscale 1, subscale 2 appears to reflect two of Deshpande, Webb and Marks 
first order factors, namely motivation and work overload, and Is labeled In 
accordance. 

The zero order correlations among the two student rating subscales, the 
first, mid-term and final examinations, and residual achievement gain for both 

1971- 72 and 1972-73 are ehown in Table 2. In the 1971-72 study, each student 
rating subscale showed strong positive relationships to the first exam, with 
steady slippage across the later exams, culminating In significant negative , 
correlations between the student ratings (positively interpreted) and residual 
achievement gain. A similar but less strong effect may be observed in the 

1972- 73 data. 

Discussion 

Examined from a research viewpoint, the rssulta of the study appear to 
be congruent With the "validity studies" reported by McKestchie, Lin and 
Hann (1971) . 



In these studlesV the relatlonshipa between etudent ratings and resi- 
dual achievement show both positive and negative signs; with the direction-* 
ality of the relationship contingent upon the type of test given the sex 
of the student , and the particular factors in the rating scale* Among these 
variables, a relatively consistent relationehipi for both sexes, is between 
^/overload'' as a factor in the rating scales and greater student performance 
on ''knowledge'^ or factual typQ tests* 

The same relationship appears to be present in the data reported here* 
The positive and negative items in subscale 1 seem to define a bipolar* 
continuum of cognitive and affective merits of the instructor with stress 
being the negative pole» In cluster tw0| work overload appears as bipolar 
to. positive motivation by the instructor as evidenced in teamworks stimulus 
variation and inspiration to effort* Interpreted in accord with the bi- 
polarity of the subscales, the results of the study suggest that the stress/ 
overload produced by the instructor is the important factor in obtaining 
greater residual gain in beginning French* Collaterally » the meritorious 
or positive behaviors of the instructor, those which jaake the course more 
enjoyable, manageable and orderly from a student viewpoint » appear to lead 
to less residual gain. Although results of this type are not very palat^blei 
their congruence with the McKeatchie data suggest that teacher behaviors which 
lead to student stress and overload be given careful attention in subsequent 
research. 

Examined from a practical viewpoint > the results of the study strong*^ 
ly suggest that student ratings of college instructois should be treated 
with great caution by college administrators and by promotion and tenure 
committees* Although such ratings may express student observations of and 
attitudes toward an ins true tor » they clearly cannoj: be routinely interpreted 
to be positive indicators of student residual achliviuent ii^ the instructor U 
course. 
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Table 1. Student Rating ItesarSubacale Correlations 



Subscale is Instructor cognitive and affective merit r with subscale 

versus student cognitive and affective stress. 
Positive Items 

41. Seemed concerned that students learn •66 

23; Put the subject across In a lively way .65 
47. Explained clearly and his explanations were 

to the point .64 

29. Made you want to do your best In tht^ course #60 
28* Gave students frequent opportunities to speak in 

French .57 
29* Provided appropriate correction and guidance in spoken 

^ wotk : •56 

30t Had a good command of French •56 
40^ Seemed to have a thorough knowledge of cultures of 

French speaking peoples V ^55 

18, Was well prepared each day ^54 

42% Arranged his presentation logically ^50 

16. Kept , course moving at an even, steady pace ' #50 
33^ Was friendly ^46 

30. Seemed sure of himself in front of the class #45 
27. Showed concern for students as persons AX 
49. Created an atmosphere In which students in the class 

seemed friendly •SS 

19* Provided time for questions and discussion tdS 
25* Expressed concepts at a level understandable by 

students i24 

Negative Items 

40. Gave vague explanations ^67 

22. Made students feel alraid of him/her #66 . 

32. Seemed dl&organl26d #63 

36. '^Talked doWiV to students j ^60 

20. Made f requent i^tonuuclation errors in French ! ^60 

24. Seemed confused in what he was doing «56 

17. Pitched his presentations above the! heads of students ^56 

31. Did not "^eem to like or understand students j .55 
15. Could not explain text materials that were coaf using 

to students I ^52 

37* Made the course unnecessarily difficult •49 

34. Showed little enthusiasm for teaching French •31 
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Table 1. continued 



Subscele 2 J Workload and motivation r with eubscale 

Positive I tend 

43. Required a reasonable amount of work ,64 

39i Inspired you to Independent efforts ,63 
Provided a variety of activities in class and used 
a variety of media (slides, films, projections, 

drawings) and outside resource persons ,62 

45. Encouraged class members to work as a team j ,56 

35* Avoided assigning a lot of burdensonu busywork .49 

Negative Ueoi 

46* Asked more than students could get done ! V ;67 
Alpha>"i64 



